Reconciling modern machine learning practice and the bias-variance trade-off

Breakthroughs in machine learning are rapidly changing science and society, yet our fundamental understanding of this technology has lagged far behind. Indeed, one of the central tenets of the field, the bias-variance trade-off, appears to be at odds with the observed behavior of methods used in the modern machine learning practice. The bias-variance trade-off implies that a model should balance under-fitting and over-fitting: rich enough to express underlying structure in data, simple enough to avoid fitting spurious patterns. However, in the modern practice, very rich models such as neural networks are trained to exactly fit (i.e., interpolate) the data. Classically, such models would be considered over-fit, and yet they often obtain high accuracy on test data. This apparent contradiction has raised questions about the mathematical foundations of machine learning and their relevance to practitioners. In this paper, we reconcile the classical understanding and the modern practice within a unified performance curve. This "double descent" curve subsumes the textbook U-shaped bias-variance trade-off curve by showing how increasing model capacity beyond the point of interpolation results in improved performance. We provide evidence for the existence and ubiquity of double descent for a wide spectrum of models and datasets, and we posit a mechanism for its emergence. This connection between the performance and the structure of machine learning models delineates the limits of classical analyses, and has implications for both the theory and practice of machine learning

Single-port endoscope-assisted resection of forehead osteoma

SummaryBackground/IntroductionEndoscope-assisted resection of forehead osteoma is a well-established procedure with the advantages of improved safety, accessibility, and visualization of the mass, avoidance of visible scarring or pigmentation on the forehead, and reduced risk of bleeding, hematoma formation, nerve injury, or paresthesia. The potential drawbacks are alopecia on the scalp incision sites and injury of the deep supraorbital nerve branch.Purpose/AimThis study aimed to evaluate the feasibility of using a single scalp access point to remove forehead osteomata.MethodsFrom 2003 to 2008, 13 patients diagnosed with forehead osteoma were retrieved from the pathology database of Taipei Veterans General Hospital, Taipei, Taiwan. Ten of the 13 patients underwent endoscope-assisted resection of the osteoma with a single scalp incision. Retrospective data collection and chart reviews were performed.ResultsThe mean age of patients undergoing the operation was 49 years. The mean size of the osteoma was 13.5 mm and the mean operative time was 27 minutes (25–30 minutes). No complications such as hematoma, alopecia, nerve injury, or infection were identified and the patients were satisfied with the esthetic results. Mean follow-up duration was 76.3 months (63–122 months).ConclusionRemoval of forehead osteoma from a single remote access with the aid of endoscopy is a safe and effective alternative. It can achieve the same esthetic and therapeutic results as the conventional two- or three-port approach without increasing the operative time or morbidities

Urbanization policy and economic development: A quantitative analysis of China's differential Hukou reforms

Published in Regional Science and Urban Economics, 2021 November, 91, Article number 103639. DOI: 10.1016/j.regsciurbeco.2020.103639</p

Covalently bonded interfaces for polymer/graphene composites

The interface is well known for taking a critical role in the determination of the functional and mechanical properties of polymer composites. Previous interface research has focused on utilising reduced graphene oxide that is limited by a low structural integrity, which means a high fraction is needed to produce electrically conductive composites. By using 4,40-diaminophenylsulfone, we in this study chemically modified high-structural integrity graphene platelets (GnPs) of 2–4 nm in thickness, covalently bonded GnPs with an epoxy matrix, and investigated the morphology and functional and mechanical performance of these composites. This covalently bonded interface prevented GnPs stacking in the matrix. In comparison with unmodified composites showing no reduction in electrical volume resistivity, the interface-modified composite at 0.489 vol% GnPs demonstrates an eight-order reduction in the resistivity, a 47.7% further improvement in modulus and 84.6% in fracture energy release rate. Comparison of GnPs with clay and multi-walled carbon nanotubes shows that our GnPs are more advantageous in terms of performance and cost. This study provides a novel method for developing interface-tuned polymer/graphene composites

Differentiable Algorithm Networks for Composable Robot Learning

This paper introduces the Differentiable Algorithm Network (DAN), a composable architecture for robot learning systems. A DAN is composed of neural network modules, each encoding a differentiable robot algorithm and an associated model; and it is trained end-to-end from data. DAN combines the strengths of model-driven modular system design and data-driven end-to-end learning. The algorithms and models act as structural assumptions to reduce the data requirements for learning; end-to-end learning allows the modules to adapt to one another and compensate for imperfect models and algorithms, in order to achieve the best overall system performance. We illustrate the DAN methodology through a case study on a simulated robot system, which learns to navigate in complex 3-D environments with only local visual observations and an image of a partially correct 2-D floor map.Comment: RSS 2019 camera ready. Video is available at https://youtu.be/4jcYlTSJF4

Development of polymer composites using modified, high-structural integrity graphene platelets

Previous studies on polymer/graphene composites have mainly utilized either reduced graphene oxide or graphite nanoplatelets of over 10 nm in thickness. In this study we covalently modified 3-nm thick graphene platelets (GnPs) by the reaction between the GnPs’ epoxide groups and the end-amine groups of a commercial long-chain surfactant (Mw = 2000), compounded the modified GnPs (m-GnPs) with a model polymer epoxy, and investigated the structure and properties of both m-GnPs and their epoxy composites. A low Raman ID/IG ratio of 0.13 was found for m-GnPs corresponding to high structural integ-rity. A percolation threshold of electrical conductivity was observed at 0.32 vol% m-GnPs, and the 0.98 vol% m-GnPs improved the Young’s modulus, fracture energy release rate and glass transition tem-perature of epoxy by 14%, 387% and 13%, respectively. These significantly improved properties are cred-ited to: (i) the low Raman ID/IG ratio of GnPs, maximizing the structural integrity and thus conductivity, stiffness and strength inherited from its sister graphene, (ii) the low thickness of GnPs, minimizing the damaging effect of the poor through-plane mechanical properties and electrical conductivity of graphene,(iii) the high-molecular weight surfactant, leading to uniformly dispersed GnPs in the matrix, and (iv) a covalently bonded interface between m-GnPs and matrix, more effectively transferring load/electron across interface

Reversible Embedding to Covers Full of Boundaries

In reversible data embedding, to avoid overflow and underflow problem, before data embedding, boundary pixels are recorded as side information, which may be losslessly compressed. The existing algorithms often assume that a natural image has little boundary pixels so that the size of side information is small. Accordingly, a relatively high pure payload could be achieved. However, there actually may exist a lot of boundary pixels in a natural image, implying that, the size of side information could be very large. Therefore, when to directly use the existing algorithms, the pure embedding capacity may be not sufficient. In order to address this problem, in this paper, we present a new and efficient framework to reversible data embedding in images that have lots of boundary pixels. The core idea is to losslessly preprocess boundary pixels so that it can significantly reduce the side information. Experimental results have shown the superiority and applicability of our work